Realistic Cloth Augmentation in Single View Video

Augmenting deformable surfaces like cloth in real video is a challenging task because not only geometric parameters describing the surface deformation have to be estimated, but also photometric parameters in order to recover realistic lighting and shading. In this paper we present a method for cloth augmentation in single-view video with realistic geometric deformation and photometric properties without a 3 dimensional reconstruction of the surface. These parameters are jointly estimated from direct image information using an extended version of optical flow with an explicit photometric color model and mesh-based. We demonstrate how the proposed method is used to produce convincing augmented results.